![]() METHOD FOR PROJECTING VIRTUAL DATA AND DEVICE FOR SAID PROJECTION
专利摘要:
The present invention relates to a method for projecting information from a digital design model comprising the following steps: a calibration step comprising an acquisition of characteristic data coming from a surface of interest, the comparison of said characteristic data; with virtual data from the digital design model, and determining a spatial position of the projection device comprising a video projector and at least two distinct image acquisition devices, and, - a projection step according to said spatial position information from the digital design model by said projector, on said surface of interest. 公开号:FR3021784A1 申请号:FR1454761 申请日:2014-05-27 公开日:2015-12-04 发明作者:Nicolas Chevassus;Denis Marraud;Antoine Tarault;Xavier Perrotton 申请人:Airbus Group SAS; IPC主号:
专利说明:
[0001] FIELD OF THE INVENTION The present invention relates to a method for projecting onto a surface of interest, virtual data from a digital design model produced by a design office. . [0002] It applies to a large number of areas, including aeronautics, construction, marine construction and space construction. The present invention applies more particularly to large objects such as aircraft, helicopters or satellites, during construction processes, assembly or maintenance. STATE OF THE ART The knowledge of the state of a real object during manufacture or in service is often partial. An industrialist usually has collections of measurements in file folders (for example spreadsheets, photos or measurement files) or in databases. It is therefore difficult, especially for large objects, not only to find the measurements from the real and associated with a particular area and understand where the measurements were made, but especially to connect and put them in correspondence with each other or with a three-dimensional digital design model. As a result, understanding of the state during manufacture or in service is not easy. [0003] 3021784 2 During installation, assembly and maintenance operations, an operator is required to perform numerous tasks such as executing procedures, taking measurements, performing diagnostics, performing work reports, etc. .etc. [0004] The operator generally uses paper plans or layers to understand and locate the different tasks to be performed. He will also use conventional means of measurement, such as a rule. This method of work can prove costly in time, and errors can occur (positioning error, damage, poor reference, etc.). One way to accomplish these tasks more quickly and efficiently is to project information directly to a surface of interest. The surface of interest corresponds to the surface with which the operator must interact. [0005] For example, a laser marking system can be used as has been demonstrated in US Patent Application No. US 2009/0195753 A1. This application discloses a device for projecting laser marking on the exterior of a vehicle comprising a plurality of laser projectors, an image of the exterior of the projected vehicle in a form synchronized by said laser projectors, a computer connected to the laser projectors and a central workstation connected to the laser projectors via the computer and to coordinate the images projected by the laser projectors. This information projection device 25 is however complex to implement and only displays a reduced amount of information because of the inherent limitations of the laser projection (flicker, loss of brightness). A rather similar device is disclosed in European Patent Application No. EP 1 719 580 A2. Indeed, this patent application discloses a device for projecting information onto a surface of interest by means of laser projectors. This device uses metrological transmitters which, via metrological receivers positioned on the surface of interest and on the laser projectors, to know the position and the orientation of the latter. This device, however, has the same limitations as the previous system and is therefore limited to types of information. In addition, the calibration system is long because of the need to position different metrological transmitters and receivers. Another known projection means is implemented via a video projector and a single camera. This system is difficult and time-consuming because it relies on the prior positioning of markers at precise positions on the surface of interest. Today, all known information projection solutions are unfortunately either complex, or not very flexible and mobile, or long to calibrate. As a result, these methods are inefficient and unusable in the majority of assembly operations. On the other hand, they have generally been thought of as performing a single, very specific task such as the designation of a position or only the tracking of targets. To date, therefore, there is a need to more simply and efficiently perform flexible and mobile information projections with high accuracy. DISCLOSURE OF THE INVENTION The present invention aims at overcoming all or part of the disadvantages of the prior art by proposing a device making it possible to implement a method for projecting information precisely recalibrated on the environment. [0006] For this purpose, according to a first aspect, the present invention relates to a method for projecting information from a digital design model comprising the following steps: a calibration step comprising an acquisition of characteristic data originating from a surface of interest, comparing said characteristic data with virtual data from the digital design model, and determining a spatial position of the projection device comprising a video projector and at least two separate image acquisition devices 10 and a projection step according to said spatial position of information from the average numerical design model of said projector on said surface of interest. [0007] The characteristic data represents a virtual reconstruction of the three-dimensional structure of the area of interest obtained by the correlation of the data acquired by each of said image acquisition devices. Thanks to these provisions, the invention makes it possible to analyze a three-dimensional structure of a surface of interest and to guide an operator responsible for carrying out assembly or control operations on said surface of interest. In the case of a mounting operation the invention would for example project the footprint of equipment to install on a part. In the case of a control operation, the invention would for example make it possible to project the three-dimensional structure of the digital design model onto the three-dimensional structure of the surface of interest, thereby enabling the operator to see the differences between the two structures. . Moreover, the spatial determination of the characteristic points of the surface of interest on said digital design model is facilitated because the invention makes it possible to dispense with a step of positioning markers at precise positions on the surface of the device. interest in order to locate on a digital design model, the spatial position of the information related to the surface of interest relative to the spatial position of said markers. This marker positioning step is often a source of errors as to the accuracy of the positioning. [0008] In embodiments, the calibration step comprises the following sub-steps: projection of calibration patterns on said surface of interest, analysis of said calibration patterns and creation of a set of correspondences between the acquisition devices of images and the video projector, and determination of the spatial position of the image acquisition devices and the video projector according to a maximum of correspondences. These embodiments make it possible to precisely determine the spatial position of the video projector relative to the image acquisition devices and then to determine the spatial position of the projection device with respect to the surface of interest. In embodiments, the calibration step is carried out continuously as follows: projection in addition to the numerical design model and in continuation of said calibration and calibration calibration patterns 20 on the surface of interest, analysis of the patterns by the image acquisition devices and continuously determining the spatial position of said image acquisition devices and the video projector. In particular, these embodiments make it possible to continuously determine the spatial position of the video projector with respect to the image acquisition devices 25 and the projection device with respect to the surface of interest, more specifically, with each new extraction of 'picture. This makes it possible, for example, to overcome the problems of precision of the calibration when the device allowing the projection is mobile and when parasitic displacements occur between the image acquisition devices and the video projector. [0009] According to embodiments, the method includes steps of detecting and highlighting differences between said digital design model and reality. [0010] Embodiments of the method include steps for tracking and determining the spatial position of known moving objects in the field of view of the image acquisition devices. This facilitates the interaction of the operator with the surface of interest. [0011] According to embodiments, the method includes steps for vibration compensation and steps for alerting in case of inability to perform this compensation. This makes it possible to avoid errors due to parasitic movements of the projector and image acquisition devices, in particular by adapting the settings of said image acquisition devices. According to embodiments, the method includes steps to avoid shadowing on the surface of interest. [0012] In some embodiments, the method includes steps for performing the three-dimensional projections. This will allow a better visualization of information by the operator with active glasses. According to another aspect, the present invention is directed to a device for projecting information from a digital design model, comprising a video projector, a computer and at least two image acquisition devices and further comprising means for acquiring characteristic data from a surface of interest, comparing said characteristic data with virtual data from the digital design model, determining a spatial position of said image acquisition devices, said video projector and said projection device, and 5 - for projecting, according to said spatial position, information from the digital design model by means of said projector, on said surface of interest. The computer allows data management between the image acquisition devices, the digital design model and the video projector. In embodiments, the image acquisition devices are global shutter cameras ("global shutter" in English terminology). This has the advantage of avoiding image deformations related to the displacement of the devices. [0013] In embodiments, the virtual data projection device comprises at least two infrared illumination sources. This allows the image acquisition devices to track any collaborative mobile object having infrared targets in the field of view of the image acquisition devices. This facilitates the interaction of the operator with the surface of interest. In embodiments, the virtual data projection device comprises means adapted to adjust the settings of the image acquisition devices according to the vibrations and to trigger an alert signal when the vibrations are too great. This improves the quality of the projected image by adapting projection settings to vibration-related constraints. [0014] In embodiments, the means adapted to adapt the settings of the image acquisition devices according to the vibrations and to trigger an alert signal when the vibrations are too large comprises an inertial sensor rigidly connected to the video projector. This inertial sensor 5 makes it possible to detect the vibrations. In embodiments, the video projector is mounted on an instrumented mechanical system comprising a position sensor and possibly motorized. This allows the video projector to be repositioned so as to extend or reduce the area covered by the projection. In addition, the position resulting from the position sensor makes it possible to update the spatial position of the projection device in the digital design model. [0015] In embodiments, the virtual data projection device comprises an autonomous power system. These provisions have the advantage of allowing the device to be more mobile. In embodiments, all elements of the information projection device from a digital design model are grouped into two boxes, the first comprising the video projector and the image acquisition devices and the second comprising the computer and the power supply. This confers on the invention an advantage related to the transport and mobility of the device that will allow the operator to handle it with ease. In embodiments, the two housings in which are grouped all the elements of the projection device, are recessed on one another. [0016] BRIEF DESCRIPTION OF THE DRAWINGS The invention will be better understood by means of the description, given below for purely explanatory purposes, of one embodiment of the invention, with reference to the figures in which: Figure 1 is a flow chart showing the method according to one embodiment of the invention; and - Figure 2 illustrates a projection of virtual data according to embodiments of the invention. [0017] DETAILED DESCRIPTION OF THE EMBODIMENTS OF THE INVENTION FIG. 1 is a flow chart showing the method according to one embodiment of the invention. In a first step 11, a calibration is performed. This step 11 comprises a projection sub-step 11a with a video projector 19, calibration patterns on an area of interest 20 visible in FIG. 2. In a following sub-step 11b, at least two acquisition devices of FIG. The images will perform a characteristic data acquisition performed by analyzing the calibration patterns projected on the surface of interest 20. Indeed, the image acquisition devices 18 capture the rendering of the pattern on the surface of interest. and said rendering is then analyzed by computer 21 to be expressed as a set of characteristic data forming a three-dimensional structure. In a subsequent sub-step 11c, the characteristic data is compared with the data from the digital design model, thus making it possible, via a vision algorithm, to create a set of correspondences between the acquisition devices of the device. 18 and the video projector 19. Finally, in a sub-step 11d, the spatial position of the video projector 19 with respect to the image acquisition devices 18 is determined according to a maximum of correspondence that said algorithm has established. between them. The spatial position of the projection device 22, comprising the video projector 19 and the image acquisition devices 18, with respect to the area of interest, is then determined. In a last step 12, according to the spatial position of the projection device 22 determined, information 5 associated with the virtual data from the digital design model is projected on the surface of interest 20. This projection of virtual data is accurate because it comes from of the digital design model. This accuracy in projected virtual data facilitates assembly or diagnostic operations of the operator. The operator then accurately detects the differences between said digital design model and the reality as they are highlighted by this projection. The calibration can be continuously optimized by projection, in addition to the digital design model and continuously, calibration patterns by the video projector 19 on the surface of interest 20. These patterns are then captured by the acquisition devices. 18 and the transformation between said image acquisition devices 18 and the video projector 19 will then be optimized continuously, that is to say, each new image extracted. [0018] A device 22 for carrying out the steps of the method 10 according to one embodiment of the invention is shown in FIG. 2. This device 22 comprises a video projector 19 and at least two image acquisition devices 18. The presence of two image acquisition devices 18 connected to a computer 21 makes it possible to obtain a three-dimensional reconstruction of the surface of interest 20 which is picked up by the image acquisition devices. The computer 21 comprises a processor and a memory and makes it possible to make the link between the characteristic points recorded on the surface of interest 20 and the digital design model. Projector 19 is used to project the virtual data directly onto the surface of interest at the precise positions at which an operation is to be performed. When projecting an image with a single light source, problems of shading areas are often encountered on the surface of interest 20 which is projected due to the relief of the surface of interest 20. The shading problems on the surface of interest can be overcome by using two to n devices 22.
权利要求:
Claims (16) [0001] REVENDICATIONS1. Method for projecting information from a digital design model characterized in that it comprises the following steps: a calibration step comprising an acquisition of characteristic data coming from a surface of interest, the comparison of said characteristic data with virtual data from the digital design model, and determining a spatial position of the projection device comprising a video projector and at least two distinct image acquisition devices, and, - a projection step according to said spatial position information from the digital design model by said projector, on said surface of interest. [0002] 2. Method according to claim 1, wherein the calibration step comprises the following sub-steps: projection of calibration patterns on said surface of interest, analysis of said calibration patterns and creation of a set of correspondences between the devices. of image acquisition and the video projector, and determination of the spatial position of the image acquisition devices and the video projector according to a maximum of correspondences. [0003] 3. Method according to claim 1 or 2, wherein the calibration step is carried out continuously as follows: projection in addition to the digital design model and continuously said calibration patterns and calibration calibration on the surface of interest, pattern analysis by the image acquisition devices and continuous determination of the spatial position of said image acquisition devices and the video projector. [0004] 4. Method according to one of the preceding claims, comprising 5 steps of detecting and highlighting differences between said digital design model and reality. [0005] The method of one of the preceding claims, comprising steps for tracking and determining spatial position of known moving objects in the field of view of the image acquisition devices. [0006] 6. Method according to one of the preceding claims, comprising steps for vibration compensation and steps for alerting in case of inability to perform this compensation. [0007] 7. Method according to one of the preceding claims, comprising steps to avoid shadowing on the surface of interest. 20 [0008] 8. Method according to one of the preceding claims, comprising steps for performing projections in three dimensions. [0009] 9. Apparatus for projecting information from a digital design model characterized in that it comprises a video projector, a computer and at least two image acquisition devices and further comprises means for: - acquiring characteristic data from a surface of interest, - comparing said characteristic data with virtual data from the digital design model, - determining a spatial position of said image acquisition devices, said projector and said device projection, and, - to project, according to said spatial position, information from the digital design model by means of said projector, on said surface of interest. [0010] 10. Device according to claim 9, wherein the image acquisition devices are global shutter cameras ("global shutter" in English terminology). [0011] 11. Device according to one of claims 9 or 10, comprising at least two infrared illumination sources. 15 [0012] 12. Device according to one of claims 9, 10 or 11, comprising means adapted to adjust the settings of the image acquisition devices according to the vibration and to trigger an alert signal when the vibrations are too important. 20 [0013] 13. Device according to claim 12, wherein the means adapted to adapt the settings of the image acquisition devices according to the vibrations and to trigger an alert signal when the vibrations are too large comprises an inertial sensor rigidly connected to the video projector. 25 [0014] 14. Device according to one of claims 9, 10, 11, 12 or 13, wherein the video projector is mounted on an instrumented mechanical system comprising a position sensor and optionally motorized. 30 [0015] 15. Device according to one of claims 9, 10, 11, 12, 13 or 14, comprising an autonomous power supply system. 3021784 15 [0016] 16. Device according to one of claims 9, 10, 11, 12, 13, 14 or 15, in which all the elements of the device are grouped together in two housings, the first comprising the video projector and the image acquisition devices and the second includes the computer and the power supply. 10 15 20
类似技术:
公开号 | 公开日 | 专利标题 EP2950235A1|2015-12-02|Method for projecting virtual data and device enabling said projection EP2103745A1|2009-09-23|Method and system for automatically calibrating earthmoving machines CA2770597A1|2011-02-24|Strain gauge, and system for spatially locating such gauges US20170003393A1|2017-01-05|Positioning Device for an Optical Triangulation Sensor JP2014169947A|2014-09-18|Shape inspection method and device thereof US20170109894A1|2017-04-20|Kinematic Data Extraction from Technical Videography FR3036473A1|2016-11-25|METHOD FOR MEASURING THE POSITION OF A MOBILE STRUCTURE EP1168831A1|2002-01-02|Method and camera calibration device EP2732238B1|2019-04-10|Method of representing possible movements of a structure for an apparatus of smartphone type FR2730058A1|1996-08-02|METHOD FOR NON-DESTRUCTIVE CONTROL OF A SURFACE, ESPECIALLY IN A HOSTILE ENVIRONMENT EP3257755B1|2019-01-23|Method for adjusting the positioning of elements of a structure of an aircraft and device for implementing same FR3088308A1|2020-05-15|METHOD FOR MEASURING A LEVEL OF WEAR OF A VEHICLE TIRE. WO2018229391A1|2018-12-20|Platform for controlling and tracking inspections of surfaces of objects by inspection robots and inspection system implementing such a platform FR2997926A1|2014-05-16|SYSTEM AND METHOD FOR AUTOMATICALLY MONITORING THE CONFORMITY OF MAKING ELEMENTS IN AN AIRCRAFT STRUCTURE FR2807603A1|2001-10-12|Method and equipment for picking up visible characteristics on walls, comprises radio controlled model helicopter with camera and means of identifying spatial position of each picture taken EP3182374A1|2017-06-21|Automated method for three-dimensional optical measurement FR3067841B1|2019-07-05|SYSTEM AND METHOD FOR LOCATING IMAGE PROCESSING EP3018625A1|2016-05-11|Method for calibrating a sight system EP3274961A1|2018-01-31|Imaging of a macroscopic plant object WO2019020924A1|2019-01-31|Surface inspection method WO2014095760A1|2014-06-26|Method for the evaluation of the metrological performance of a measurement of a kinematic field by correlation of digital images FR3093215B1|2021-08-27|Method and device for monitoring the environment of a robot FR3015033A1|2015-06-19|METHOD AND DEVICE FOR ANALYZING THE SURFACE OF A SUBSTRATE FR2997492A1|2014-05-02|Verification plan establishment method for e.g. water pipe network, involves rebuilding apparent element, and establishing plan including apparent element that is re-built and geo-referred in predetermined geometrical reference frame KR100516073B1|2005-09-21|Jig for measuring 3D non-contact type
同族专利:
公开号 | 公开日 SG10201503994UA|2015-12-30| JP2016015720A|2016-01-28| CA2891149A1|2015-11-27| US10044996B2|2018-08-07| BR102015012167A2|2015-12-29| EP2950235A1|2015-12-02| US20150350617A1|2015-12-03| CN105353999B|2019-05-10| CN105353999A|2016-02-24| JP6591200B2|2019-10-16| FR3021784B1|2017-10-13| RU2015119957A|2016-12-20|
引用文献:
公开号 | 申请日 | 公开日 | 申请人 | 专利标题 US20090086199A1|2007-09-28|2009-04-02|The Boeing Company|Method involving a pointing instrument and a target object| EP2400261A1|2010-06-21|2011-12-28|Leica Geosystems AG|Optical measurement method and system for determining 3D coordination in a measuring object surface| WO2012136345A2|2011-04-04|2012-10-11|EXTEND3D GmbH|System and method for visually displaying information on real objects| US20140015963A1|2012-07-15|2014-01-16|8 Tree Llc|Portable three-dimensional metrology with data displayed on the measured surface|WO2020058643A1|2018-09-21|2020-03-26|Diotasoft|Method, module and system for projecting onto a workpiece an image calculated on the basis of a digital mockup|JP2575910B2|1990-02-19|1997-01-29|株式会社電通プロックス|Automatic tracking projector| JPH04157005A|1990-10-15|1992-05-29|Nkk Corp|Method for detecting generation of thickness deviation of seamless tube| US6969960B2|1999-09-10|2005-11-29|Belliveau Richard S|Image projection lighting device| US7710391B2|2002-05-28|2010-05-04|Matthew Bell|Processing an image utilizing a spatially varying pattern| JP2003343552A|2002-05-27|2003-12-03|Koyo Seiko Co Ltd|Cone roller bearing| US6811264B2|2003-03-21|2004-11-02|Mitsubishi Electric Research Laboratories, Inc.|Geometrically aware projector| EP1682936B1|2003-09-10|2016-03-16|Nikon Metrology NV|Laser projection systems and method| US20080229254A1|2006-03-24|2008-09-18|Ervin-Dawson Warner|Method and system for enhanced cursor control| US7887191B2|2008-02-01|2011-02-15|The Boeing Company|Method and system for making a large object itilizing laser projection| US8761434B2|2008-12-17|2014-06-24|Sony Computer Entertainment Inc.|Tracking system calibration by reconciling inertial data with computed acceleration of a tracked object in the three-dimensional coordinate system| US8786682B2|2009-03-05|2014-07-22|Primesense Ltd.|Reference image techniques for three-dimensional sensing| US20100309340A1|2009-06-03|2010-12-09|Border John N|Image sensor having global and rolling shutter processes for respective sets of pixels of a pixel array| EP2433716A1|2010-09-22|2012-03-28|Hexagon Technology Center GmbH|Surface spraying device with a nozzle control mechanism and a corresponding method| EP2939423A1|2012-12-28|2015-11-04|Metaio GmbH|Method of and system for projecting digital information on a real object in a real environment|FR3022065B1|2014-06-04|2017-10-13|European Aeronautic Defence & Space Co Eads France|METHOD FOR GENERATING AN ENRICHED DIGITAL MODEL| ES2686470T3|2015-12-10|2018-10-18|Airbus Defence and Space GmbH|Modular device for high speed video vibration analysis| CN107817003B|2016-09-14|2021-07-06|西安航通测控技术有限责任公司|External parameter calibration method of distributed large-size space positioning system| DE102017202652A1|2017-02-20|2018-08-23|Siemens Aktiengesellschaft|Apparatus and method for calibration by means of projected patterns| DE102017202651A1|2017-02-20|2018-08-23|Siemens Aktiengesellschaft|Apparatus and method for calibration by means of projected virtual level patterns| CN107463902B|2017-08-07|2020-11-24|上海碧虎网络科技有限公司|Method and system for automatically identifying projection abnormity of projector| DE102017010741B4|2017-11-21|2021-01-14|Diehl Aerospace Gmbh|Procedure for setting up a projector, projector and passenger cabin| WO2019173808A1|2018-03-09|2019-09-12|Ventana 3D, Llc|Real-time quality control during manufacturing using augmented reality| CN111182278A|2018-11-09|2020-05-19|上海云绅智能科技有限公司|Projection display management method and system| CN109801274B|2019-01-09|2022-03-01|江南造船有限责任公司|Projection-based part assembling method and device, equipment and storage medium| CN110769237A|2019-11-01|2020-02-07|中航通用电气民用航电系统有限责任公司|Projection system, apparatus and method for projection position adjustment using head position|
法律状态:
2015-05-21| PLFP| Fee payment|Year of fee payment: 2 | 2015-12-04| PLSC| Publication of the preliminary search report|Effective date: 20151204 | 2016-05-20| PLFP| Fee payment|Year of fee payment: 3 | 2017-05-23| PLFP| Fee payment|Year of fee payment: 4 | 2017-07-28| CD| Change of name or company name|Owner name: AIRBUS GROUP SAS, FR Effective date: 20170622 | 2017-07-28| CA| Change of address|Effective date: 20170622 | 2017-09-29| TP| Transmission of property|Owner name: AIRBUS, FR Effective date: 20170821 | 2018-05-22| PLFP| Fee payment|Year of fee payment: 5 | 2019-05-22| PLFP| Fee payment|Year of fee payment: 6 | 2020-05-22| PLFP| Fee payment|Year of fee payment: 7 | 2021-05-20| PLFP| Fee payment|Year of fee payment: 8 |
优先权:
[返回顶部]
申请号 | 申请日 | 专利标题 FR1454761A|FR3021784B1|2014-05-27|2014-05-27|METHOD FOR PROJECTING VIRTUAL DATA AND DEVICE FOR SAID PROJECTION|FR1454761A| FR3021784B1|2014-05-27|2014-05-27|METHOD FOR PROJECTING VIRTUAL DATA AND DEVICE FOR SAID PROJECTION| EP15167279.7A| EP2950235A1|2014-05-27|2015-05-12|Method for projecting virtual data and device enabling said projection| CA2891149A| CA2891149A1|2014-05-27|2015-05-13|Method for projecting virtual data and device allowing this projection| US14/719,307| US10044996B2|2014-05-27|2015-05-21|Method for projecting virtual data and device enabling this projection| SG10201503994UA| SG10201503994UA|2014-05-27|2015-05-21|Method for projecting virtual data and device enabling this projection| RU2015119957A| RU2015119957A|2014-05-27|2015-05-26|METHOD FOR PROJECTING VIRTUAL DATA AND DEVICE FOR SUCH PROJECTION| BR102015012167A| BR102015012167A2|2014-05-27|2015-05-26|projection process of virtual data and device, allowing this projection| CN201510275743.4A| CN105353999B|2014-05-27|2015-05-26|Equipment of the method with this projection is enabled for projected virtual data| JP2015106162A| JP6591200B2|2014-05-27|2015-05-26|Virtual data projection method and projection apparatus| 相关专利
Sulfonates, polymers, resist compositions and patterning process
Washing machine
Washing machine
Device for fixture finishing and tension adjusting of membrane
Structure for Equipping Band in a Plane Cathode Ray Tube
Process for preparation of 7 alpha-carboxyl 9, 11-epoxy steroids and intermediates useful therein an
国家/地区
|